Liquid Time-constant Networks

We introduce a new class of time-continuous recurrent neural network models. Instead declaring learning system's dynamics by implicit nonlinearities, we construct networks linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent with varying (i.e., liquid) time-constants coupled to their hidden state, outputs being computed numerical differential equation solvers. These exhibit stable and bounded behavior, yield superior expressivity within the family ordinary equations, give rise improved performance on time-series prediction tasks. To demonstrate these properties, first take theoretical approach find bounds over dynamics, compute expressive power trajectory length measure in latent space. then conduct series experiments manifest approximation capability Liquid Time-Constant Networks (LTCs) compared classical modern RNNs.